Skip to content

Upgrade @huggingface/transformers to v4.0.0-next.8 for Qwen3.5 support - #59

Merged
devlux76 merged 2 commits into
mainfrom
copilot/fix-qwen35-handler
Mar 20, 2026
Merged

Upgrade @huggingface/transformers to v4.0.0-next.8 for Qwen3.5 support#59
devlux76 merged 2 commits into
mainfrom
copilot/fix-qwen35-handler

Conversation

Copilot AI commented Mar 20, 2026

Copy link
Copy Markdown
Contributor

v3.8.1 (latest) does not support the qwen3_5 model type or its TokenizersBackend tokenizer class, causing hard failures when loading onnx-community/Qwen3.5-0.8B-ONNX (the default chat and T4 benchmark model).

Root cause

  • Unsupported model type: qwen3_5AutoModelForCausalLM did not register Qwen3_5ForConditionalGeneration until v4
  • Unknown tokenizer class "TokenizersBackend" — requires the new @huggingface/tokenizers peer dep introduced in v4

Changes

  • package.json@huggingface/transformers: latest4.0.0-next.8
  • bun.lock — updated transitive dependency tree:
    • Added @huggingface/tokenizers@0.1.3 (new in v4)
    • onnxruntime-web: 1.22.0-dev1.25.0-dev.20260307-d626b568e0
    • onnxruntime-node: 1.21.01.24.3
    • onnxruntime-common: 1.21.01.24.3
    • Added adm-zip@0.5.16 (new dep of onnxruntime-node 1.24.3, replaces tar)

No changes to worker.ts — the v4 public API (pipeline, TextStreamer, InterruptableStoppingCriteria, env, TextGenerationPipeline) is fully backward-compatible with the existing call sites.

Original prompt

This section details on the original issue you should resolve

<issue_title>Qwen3.5 may need specific handling, also we are on an older stale transformers.js</issue_title>
<issue_description>Please review the docs for https://huggingface.co/onnx-community/Qwen3.5-0.8B-ONNX
I think we might need to be on transformers@next and we might need to using a qwen3.5 specific function. We should probably check all models for this sort of shenanigans.
Here is the section from the page in case it's inaccessible.

const model_id = "onnx-community/Qwen3.5-0.8B-ONNX";
const processor = await AutoProcessor.from_pretrained(model_id);
const model = await Qwen3_5ForConditionalGeneration.from_pretrained(model_id, {
  dtype: {
    embed_tokens: "q4",
    vision_encoder: "fp16",
    decoder_model_merged: "q4",
  },
  device: "webgpu",
});

Here is the error you need to fix now

worker.ts:264 Unknown tokenizer class "TokenizersBackend", attempting to construct from base class.
from_pretrained @ transformers.web.js:30347
await in from_pretrained
loadItems @ transformers.web.js:25826
pipeline @ transformers.web.js:25764
loadModel @ worker.ts:264
await in loadModel
(anonymous) @ worker.ts:452Understand this warning
worker.ts:156 [q2 worker] 2026-03-20T08:29:40.924Z [warn] Backend failed {device: 'wasm', error: Error: Unsupported model type: qwen3_5
    at AutoModelForCausalLM.from_pretrained (https://devlux7…}
workerLog @ worker.ts:156
loadModel @ worker.ts:316
await in loadModel
(anonymous) @ worker.ts:452Understand this warning
worker.ts:158 [q2 worker] 2026-03-20T08:29:40.924Z [error] All backends failed, sending error message to main thread {lastErr: Error: Unsupported model type: qwen3_5
    at AutoModelForCausalLM.from_pretrained (https://devlux7…}
```</issue_description>

<agent_instructions>We should probably audit the codebase entirely and make sure we're using the transformers@next version of things and also ensure we're using the correct functions throughout.</agent_instructions>

## Comments on the Issue (you are @copilot in this section)

<comments>
</comments>

📍 Connect Copilot coding agent with Jira, Azure Boards or Linear to delegate work to Copilot in one click without leaving your project management tool.

…en3.5 support

Fixes two errors when loading onnx-community/Qwen3.5-0.8B-ONNX:
- Unknown tokenizer class "TokenizersBackend" (v4 adds @huggingface/tokenizers dep)
- Unsupported model type: qwen3_5 (v4 adds qwen3_5 model type support)

Updates bun.lock with all new transitive dependencies:
- @huggingface/tokenizers@0.1.3 (new)
- onnxruntime-web@1.25.0-dev.20260307-d626b568e0
- onnxruntime-node@1.24.3
- onnxruntime-common@1.24.3
- adm-zip@0.5.16 (new dep of onnxruntime-node 1.24.3)

All 250 tests pass. TypeScript and ESLint clean.

Co-authored-by: devlux76 <86517969+devlux76@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix specific handling for Qwen3.5 and update transformers.js Upgrade @huggingface/transformers to v4.0.0-next.8 for Qwen3.5 support Mar 20, 2026
Copilot AI requested a review from devlux76 March 20, 2026 08:52

@devlux76 devlux76 left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@devlux76
devlux76 marked this pull request as ready for review March 20, 2026 16:19
Copilot AI review requested due to automatic review settings March 20, 2026 16:19

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Upgrades the app’s @huggingface/transformers dependency to a v4 “next” release to enable loading/running Qwen3.5 ONNX models (the repo’s default model), addressing the current hard failures around the qwen3_5 model type and tokenizer backend support.

Changes:

  • Upgrade @huggingface/transformers from latest/v3.x to 4.0.0-next.8.
  • Update bun.lock to the new dependency graph, including new @huggingface/tokenizers and updated onnxruntime-* packages.

Reviewed changes

Copilot reviewed 1 out of 2 changed files in this pull request and generated 1 comment.

File Description
package.json Pins the project to a transformers.js v4 prerelease needed for Qwen3.5 support.
bun.lock Locks updated transitive dependencies introduced by the transformers.js v4 upgrade (tokenizers + onnxruntime updates).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread package.json
},
"dependencies": {
"@huggingface/transformers": "latest"
"@huggingface/transformers": "^4.0.0-next.8"

Copilot AI Mar 20, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using a caret range with a pre-release tag ("^4.0.0-next.8") can unintentionally advance to newer pre-releases or the eventual stable 4.0.0 on the next install/lock refresh, which is a meaningful runtime risk for this app. Consider pinning to the exact version ("4.0.0-next.8") if you want deterministic behavior until you explicitly choose to upgrade again.

Suggested change
"@huggingface/transformers": "^4.0.0-next.8"
"@huggingface/transformers": "4.0.0-next.8"

Copilot uses AI. Check for mistakes.
@devlux76
devlux76 merged commit 71d84ff into main Mar 20, 2026
7 checks passed
@devlux76
devlux76 deleted the copilot/fix-qwen35-handler branch March 20, 2026 16:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Qwen3.5 may need specific handling, also we are on an older stale transformers.js

3 participants